Interactive entertainment system for presenting supplemental interactive content together with continuous video programs

ABSTRACT

An interactive entertainment system enables presentation of supplemental interactive content along side traditional broadcast video programs, such as television shows and movies. The programs are broadcast in a conventional manner. The supplemental content is supplied as part of the same program signal over the broadcast network, or separately over another distribution network. A viewer computing unit is located at the viewer&#39;s home to present the program and supplemental content to a viewer. When the viewer tunes to a particular channel, the viewer computing unit consults an electronic programming guide (EPG) to determine if the present program carried on the channel is interactive. If it is, the viewer computing unit launches a browser. The browser uses a target specification stored in the EPG to activate a target resource containing the supplemental content for enhancing the broadcast program. The target resource contains display layout instructions prescribing how the supplemental content and the video content program are to appear in relation to one another when displayed. When the data from the target resource is downloaded, the viewer computing unit is responsive to the layout instructions obtained from the target resource to display the supplemental content concurrently with the video content program. Embedding the layout instructions in the supplemental content advantageously places control of the presentation to the content developers. The developers are free to arrange the data and video in any manner they choose.

RELATED APPLICATIONS

This is a divisional of, and incorporates by reference, U.S. patentapplication Ser. No. 09/349,638, filed Jul. 8, 1999, which is acontinuation of U.S. Pat. No. 08/623,610 now U.S. Pat. No. 6,240,555,filed Mar. 29, 1996.

TECHNICAL FIELD

This invention relates to interactive entertainment systems, such asinteractive television or interactive computing network systems. Moreparticularly, this invention relates to methods for presentingsupplemental interactive content in conjunction with continuous videoprograms to enable viewer interactivity with the otherwisenon-interactive video programs.

BACKGROUND OF THE INVENTION

Video content programs, such as TV shows and pay-per-view movies, aredelivered to a viewer as a continuous data stream. Today, programs aremost commonly distributed using a wireless broadcast system, or a cablesystem. In the first instance, the programs are broadcast over awireless network and received at individual homes through an antenna orsatellite dish. In the latter case, the programs are transmitted overcable to set-top boxes resident in the viewers' homes.

In broadcast distribution systems, there is no opportunity forinteractive control of the content by the viewer. The viewer simply hasan option to watch the program, change to a different program, or turnoff the television. However, as consumers have learned from playingvideo games on their televisions, non-interactive viewing is not nearlyas fun or sensory rich as interactive entertainment.

To enhance the traditional way of viewing television, there has beensome effort toward the production of interactive programming content. Aspresently contemplated, additional interactive content is created toenhance the existing traditional program. This supplemental content isplayed along with the continuous video stream to enable viewers tointeract with the program in a more involved manner than simply watchingit. The supplemental content might, for example, ask the viewerquestions about the episode, or play games with the viewer that relateto the show, or describe behind-the-scenes aspects of making theprogram. However, there remains a significant hurdle concerning how tocreate and distribute interactive programs.

Apart from the TV environment, however, computer users are familiar withinteractive content on their computers. Many users own interactivemultimedia CD-ROM applications that combine video, audio, pictures,text, and other content into a rich and responsive presentation.Familiar examples of interactive computer applications include games(e.g., Myst from Broderbund), educational programs (e.g., Reader Rabbitseries from the Learning Company or Magic Schoolbus series fromMicrosoft), and home entertainment (e.g., Encarta from Microsoft).

Interactive content is also available from online services over a publicnetwork. Most notably, the Internet is emerging as a means for supplyingvideo, sound, pictures, text, and other multimedia rich resources to auser's computer. Through the Internet, users can access a wide varietyof resources that are maintained on computers located around the world.

Resources available on the Internet are most commonly presented ashypertext. “Hypertext,” also referred to as “hypermedia,” is a metaphorfor presenting information in which text, images, sounds, and actionsbecome linked together in a complex, non-sequential web of associationsthat permit a user to browse through related topics, regardless of thepresented order of the topics. Hypermedia content is widely used fornavigation and information dissemination on the “World-Wide Web” (WWW orWeb) of the Internet. An application program referred to as a “Webbrowser” is normally used to retrieve and render hypermedia content fromthe WWW.

Despite the development of interactive entertainment in the computersector, there has been little activity spilling over into thetraditional television world. The TV environment poses a problem in thatthe programs are typically delivered as a broadcast of a continuous datastream, which inherently does not support interactive control.Interlacing interactive content presents a difficult design issue.

One proposed solution is a technology referred to as “Intercasttechnology” which is available from the Intercast Industry Group, aconsortium of leading television networks, program hardware vendors, andsoftware vendors. The Intercast technology presents both televisionprograms and Internet data together on the same television or computermonitor, but with separate and predefined panes.

FIG. 1 shows a screen 14 (television or computer monitor) whichillustrates the Intercast technology. The screen 14 is divided intopanes, as represented by panes 16-18. Pane 16 contains the televisionprogram resulting from the video data. Pane 17 contains a hypermediadocument, such as a Web page, that is provided by the Internet data. Athird pane 18 can be used to show additional data, such asadvertisements or the like.

The Internet data is combined with the video data of the televisionprogram to form a single signal that is broadcast to the viewer. TheInternet data is transmitted during the vertical blank interval (VBI)between successive frames of the video data. The Internet data and videodata are separated at the viewer's computer and presented simultaneouslywithin their respective panes.

The drawback with the Intercast technology is that it rigidly adheres tothe paned presentation. The television pane 16 is a self-contained panewhich is dedicated to showing only the video program, and the Web pane17 is a separately self-contained pane which is reserved exclusively forWeb content. Content providers who develop the Internet data have nocontrol over how the television program and interactive supplementalcontent is presented to the viewer. They can simply control how the datais presented within its own box 17. Accordingly, the content providersare significantly limited in what they can create in the way of a fullinteractive media event.

The inventors have developed a better way of creating and distributinginteractive programming that frees the content providers of theserestrictions.

SUMMARY OF THE INVENTION

This invention concerns an interactive entertainment system forsupplying interactive supplemental content along with continuous videocontent programs to viewers. The programs are supplied from a programprovider, such as a cable headend or a broadcast station, over adistribution network to a viewer computing unit. The programs arecontinuous, non-interactive data streams, such as television shows,movies, or other video content.

The viewer computing unit is implemented as a television, atelevision/set-top box unit, a personal computer, or the like. It has aprocessor and memory. An electronic programming guide (EPG) is stored inthe memory and executable on the processor to organize programminginformation that is descriptive of the video content programs. The EPGmaintains a data field that indicates whether the video content programis interactive. The EPG data field contains a pointer, universalresource locator, or other target specification to the target resourcethat supports the interactive content and correlates this data fieldwith the particular program that the supplemental content is designed toenhance. Accordingly, if a target resource is listed in conjunction witha particular program, the program is recognized as being interactive.The supplemental content can be developed and provided by the sameprovider that distributes the video content program, or by anindependent service provider.

When the viewer tunes to a particular channel, the viewer computing unitconsults the EPG to determine if the present program is interactive. Ifit is, the viewer computing unit launches an interactive support module,such as an Internet browser. This browser is kept in memory and isdynamically loadable for execution on the processor when the viewertunes to a channel carrying a video content program that the EPGidentifies as interactive. The viewer computing unit also depicts asmall icon or other indicia to alert the viewer that the program isinteractive. The viewer can click on or otherwise activate the icon toenter the interactive mode and display the supplemental content. As analternative, the supplemental content can be automatically displayed inresponse to launching the Internet browser.

The Internet browser uses the target specification in the EPG to startthe target resource. The target resource contains the supplementalcontent to enhance the television program. The supplemental contentmight be, for example, questions about the program, games, triviainformation, facts about the actors and producers, information on otherepisodes, advertisements, a listing of products or memorabilia about theprogram, and so on. The supplemental content and program can betransmitted together as a single signal, which is separated at theviewer computing unit, or separately over two channels or two differentnetworks.

The target resource also contains display layout instructionsprescribing how the supplemental content and the video content programare to appear in relation to one another when displayed on thetelevision or monitor. When the data from the target resource isdownloaded to the viewer computing unit, the processor is responsive tothe layout instructions obtained from the target resource to display thesupplemental content concurrently with the video content program.

Embedding the layout instructions in the supplemental contentadvantageously returns control of the presentation to the contentdevelopers. The developers can now arrange the data and video in anymanner they choose. The developers are not restricted to specific fixedpanes, as in the prior art.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is an illustration of a screen showing a paned presentationformat for presenting television programs and Internet data according toprior art Intercast technology.

FIG. 2 is a diagrammatic illustration of an interactive entertainmentsystem according to one implementation. The FIG. 2 implementation isrepresentative of an interactive TV system.

FIG. 3 is a simplified example of data fields in a data structuremaintained by an electronic programming guide.

FIG. 4 is a diagrammatic illustration of an interactive entertainmentsystem according to a second implementation, which is representative ofan interactive integrated PC-TV system.

FIG. 5 is a block diagram of a viewer computing unit according to anaspect of this invention.

FIGS. 6 and 7 show a flow chart for operating an interactiveentertainment system.

FIG. 8 a-8 c are a series of exemplary screen illustrations showingdifferent display layouts of the video program and the supplementalcontent.

FIG. 9 is a flow chart of a method for authoring an interactiveentertainment program.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

FIG. 2 shows an interactive entertainment system 20 implemented as aninteractive television (ITV) system. System 20 includes a centralizedheadend 22 which is configured to provide continuous video contentprograms to multiple subscribers. These programs might includetraditional broadcast TV shows, movies, games, and the like. Theprograms are transmitted as a continuous data stream, which isnon-interactive in itself.

Each subscriber has a viewer computing unit 24. In the illustratedimplementation, the viewer computing unit is embodied as a set-top box(STB) 26 coupled to a television (TV) 28. The STB 26 receives digitalvideo signals from headend 22 and controls which programs are displayedon the TV 28. The STB 26 is controlled by the viewer through use of apanel control located on its face, or a hand held remote control unit30. Instead of a separate STB, however, a viewer computing unit can beincorporated in the TV itself. In addition to televisions, the viewercomputing unit might be implemented with other types of visual displayunits, such as a computer monitor, as is described below with respect tothe FIG. 4 implementation.

Each viewer computing unit 24 is configured to run an electronic programguide (EPG). The EPG provides an on-screen listing of various programsor program titles. The listing is organized in a predeterminedarrangement that is displayed on the television. The EPG might alsoinclude other program descriptive information, including whether theprogram is provided in closed caption or stereo.

Headend 22 is interconnected to the subscribers via a distributionstructure or network 32. The distribution structure 32 can beimplemented in different ways. One implementation is a multi-tiernetwork which includes a high-speed, high-bandwidth fiber optic cablenetwork between the headend and regional distribution nodes (not shown),and conventional home entry lines, such as twisted-pair lines or coaxialcable, between the distribution nodes and viewer computing units. Otherimplementations include satellite communications (e.g., DSStechnologies), RF communication, or other wireless technologies.Moreover, the network can be constructed using a combination of wirelessand wire-based technologies.

In the illustrated implementation, the network 32 provides both thepipeline for distribution of programs from the headend to thesubscriber, as well as a back channel which enables communication fromthe subscriber to the headend. The back channel permits the viewer tosend requests and instructions to the headend to facilitate interactivecontrol.

Headend 22 provides video content programs to the viewer computing unit24. The programs are embodied as video data streams that are transmittedfrom headend 22 over distribution structure 32 to the viewer computingunit. The headend maintains a database of programs 40 that can be servedby a continuous media server 42 to individual viewers in an on-demandmode. The headend can also retransmit to its subscribers broadcast videosignals that it receives from another source, such as a satellite feedor another cable system. The video data streams are sent to the viewercomputing units in digital or analog format.

An EPG server 44 is provided at headend 22 to serve the programminginformation needed by the EPG at the viewer computing unit 24. The EPGserver 44 is implemented as a structured query language (SQL) database46 with records containing information relating to available shows orprograms.

The headend 22 further includes an enhanced content server 52 whichserves supplemental interactive content to the viewer computing units toenhance or supplement the continuous video streams served by thecontinuous media server 42. The supplemental content is stored digitallyin database 54 and can be text, graphics, video, picture, sound, orother multimedia types. Examples of supplemental content include triviaquestions or games related to the program, advertisements, merchandiseor other memorabilia, hyperlinks to similar programs of similar type orstarring the same actor/actress, and so on.

In the implementation described herein, the supplemental content isconstructed as a hypertext file which is rendered by a browser.Hypertext, or hypermedia, is a metaphor for presenting information inwhich text, images, sounds, and actions become linked together in acomplex, non-sequential web of associations that permit a user to browsethrough related topics, regardless of the presented order of the topics.Hypermedia content is widely used for navigation and informationdissemination on the “World-Wide Web” (WWW or Web) of the Internet.

Hypermedia content is commonly organized as documents with embeddedcontrol information. The embedded control information includesformatting specifications, indicating how a document is to be renderedby the browser. In addition, such control information can include linksor “hyperlinks,” which are symbols or instructions telling the browserwhere to find other related documents on the Internet. A hyperlinkincludes a label, which is typically rendered as a graphical icon or ashighlighted keywords, and an underlying target specification. The targetspecification is set forth in the underlying hypermedia document, but isnormally invisible to the user. The target specification unambiguouslyidentifies a targeted document or resource, specifying the name of thecomputer on which the document resides and the complete file name of thedocument.

In concept, the target resource can be virtually any type ofobject-including executable programs, text or multimedia documents,sound clips, audio segments, still images, computers, directories, andother hyperlinks. In WWW documents, hyperlink targets are most oftenfiles that can reside on any computers connected to the Internet.However, a hyperlink target can also be a particular location within adocument, including the document that is currently being rendered.Hypertext usage is not limited to the Internet. Various multimediaapplications utilize hypertext to allow users to navigate throughdifferent pieces of information content.

FIG. 3 shows an example data structure 48 which is used by the EPGdatabase 46 to organize programming information and to correlate targetspecifications with the programs. The data structure includes a numberof data records comprising various data fields 50 for holdingprogramming information. The data fields contain program titles, actornames, whether the program has closed captioning or stereo audio, thescheduled time of the program, the network name, and description text.Each program record in the data structure 48 also holds pointers tostorage locations within the continuous media server 42 which hold thevideo data streams of the programs corresponding to the program records.

The data structure 48 includes a data field 58 for holding targetspecifications which reference target resources supporting thesupplemental content. The data structure correlates the targetspecifications with the programs by associating them within the sameprogram record. The presence of a target specification within the datafield 58 indicates that the associated program is interactive and thatcomplementary content can be displayed in addition to the programitself. The target specification can be implemented in different forms.The target specification to the supplemental content stored at theheadend which enhances the “Murder, She Wrote” program is in the form ofa pointer “CONTENT1.” The presence of this pointer in field 58 indicatesthat the program “Murder, She Wrote” is interactive compatible and thatthe supplemental content resides at a memory location identified by thepointer “CONTENT1.” Similarly, the programs “Seinfeld” and “Star Trek”are interactive compatible and have associated supplemental contentwhich are located by the hyperlink URLs in corresponding data field 58.These URLs reference resources provided by independent service providers(described below with reference to FIG. 4).

A URL (universal resource locator) is a type of target specificationused in WWW documents to describe everything about a particular resourcethat a browser needs to know to request and render it. The URL describesthe protocol a browser should use to retrieve the resource, the name ofthe computer it is on, and the path and file name of the resource.

The following is an example of a URL listed in the EPG data structure 48in association with the “Seinfeld” program:

-   -   http://www.nbc.com/seinfeld

The “http://” portion of the URL describes the protocol. The letters“http” stand for HyperText Transfer Protocol, the set of rules that abrowser will follow to request a document and the remote server willfollow to supply the document. The “www.nbc.com” portion of the URL isthe name of the remote host computer which maintains the document. Thelast portion “/seinfeld” is the path and file name of the document onthe remote host computer.

The development of the EPG data structure 48 includes designatingseparate data fields 50 within a storage medium for holding programminginformation, and dedicating at least one of the data fields 58 tostoring target specifications to supplemental content. The targetspecifications can be in the form of memory pointers, hyperlinks, URLs,or any other designation for referencing a location containingsupplemental content. For programs with no corresponding supplementalcontent, the target specifications data field 58 is left empty. However,for those programs with supplemental content, the target specificationto the target resource containing the supplemental content is entered inthe data field 58 and correlated with the program. The correlation isprovided through the data structure by the placing the targetspecification within the same data record as the program title and otherinformation for a particular program. It is further noted that thetarget resources can be embedded within other text-based data held inother data fields 50 which also relate to the corresponding programs.

The same or a subset of data structure 48 is employed at the EPGapplication running at the viewer computing unit in the home. The datarecords stored at the headend on the EPG server are transmittedperiodically in batch, or individually, and cached at the local EPG. Thelocal EPG is thus able to identify whether a particular program isinteractive compatible by quick reference to the locally cached EPG datastructure.

FIG. 4 shows an interactive entertainment system 60 according to asecond implementation which accommodates third party independent serviceproviders. Interactive entertainment system 60 includes headend 22 whichsupplies programs and supplemental content to a viewer computing unit62. In this implementation, the viewer computing unit 62 is implementedas a computer 64 with a monitor 66, a central processing unit 68, akeyboard 70, and a mouse 72 (although other input devices can be used inaddition to, or instead of, the keyboard and mouse).

The headend 22 serves the programs over a first network 74, which may beimplemented like the fiber optic distribution structure described above,or as a satellite system or other wireless broadcast system, or as aconventional data network. It is noted that other program providers maybe used instead of a headend, such as a broadcast station or an onlineservice provider.

The interactive entertainment system 60 also includes multipleindependent service providers (ISPs), as represented by ISP 80, whichdistribute digital content to the viewer computing unit 62 over a secondnetwork 82. An example of the second network 82 is a public network,such as the Internet. The ISP 80 has a host 84 and a content database 86to serve various multimedia content to the viewer's computer. The ISPhost 84 stores one or more target resources that can be rendered by theviewer computing unit 62.

The supplemental content provided by the ISP 80 is correlated with theprograms by data structure 48 as described above. As shown in FIG. 3,the data field 58 includes target specifications to supplemental contentprovided by servers other than the program provider which distributesthe show. In the illustrated example, the programs Seinfeld and StarTrek have associated ISP-served content which are located by thehyperlink URLs in corresponding data field 58. It is further noted thata program might have more than one source of supplemental content. Theprogram record for “Star Trek” has two associated target specifications,one that references a target resource at the host computer “www.fox.com”and one that references a target resource at the host computer“www.collections.com.”

According to this FIG. 4 arrangement, the viewer computing unit 62receives a non-interactive, continuous video stream from the headend 22,and supplemental interactive content from either the headend 22 or froman independent service provider 80. The supplemental content mighttherefore be carried to the viewer's computer over the same channel asthe program, over a separate channel, or over a separate connectionunrelated to the program channels. The back channel for facilitatinginteractive control is provided through either network 72 or 82.

In another implementation, the interactive content can be suppliedlocally by a storage medium, such as a CD-ROM. Suppose, for example, acontent developer creates an interactive CD-ROM that can be played alongwith a particular movie. The supplemental content is accessed via diskreads to the local storage drive, rather than using URLs to targetresources over a network. The supplemental content is synchronized withthe program using open loop control, such as a start time followed bymeasurable ticks or by frame count.

FIG. 5 shows a viewer computing unit, referenced generally as numeral90, in more detail. The viewer computing unit 90 includes a processor92, a volatile memory 94, and a program memory 96. The viewer computingunit 90 also has at least one receiver, and possibly two receivers, forreceiving the video stream from the headend and the digital supplementaldata from the headend or ISP. The first receiver is in the form of atuner 98 which tunes to the channel or broadcast frequency to receive avideo data stream from a program source, such as the headend,broadcaster, or other program provider. The second receiver, referencedgenerally as 100, can be implemented as a second tuner for receiving thecontent over a cable or wireless distribution network, or a modem forreceiving the supplemental content over the Internet or other datanetwork.

The viewer computer runs an operating system 101 which supports multipleapplications. The operating system 101 is stored in memory and executeson the processor. The operating system is preferably a multitaskingoperating system which allows simultaneous execution of multipleapplications. The operating system 101 employs a graphical userinterface windowing environment which presents the applications ordocuments in specially delineated areas of the display screen called“windows.” One preferred operating system is a Windows® brand operatingsystem sold by Microsoft Corporation, such as Windows® 95 or Windows® NTor other derivative versions of Windows®. However, other operatingsystems which provide windowing environments may be employed, such asthe Macintosh operating system from Apple Corporation and the OS/2operating system from IBM.

A channel navigator application 102 is stored in program memory 96 andexecutes on the processor 92 to control the tuners 98 and 100 to selecta desired channel for receiving the video content programs. An EPGapplication 104 is stored in program memory 96 and executes on theprocessor 92 to organize programming information downloaded from the EPGserver at the headend. The EPG 104 supports a displayable user interface(UI) which visually correlates programs titles to scheduled viewingtimes and tuning information, such as a channel, in a scrollable gridformat. The viewer computing unit 90 also has an interactive supportmodule in the form of browser 106 which is kept in memory 96. Thebrowser 106 is dynamically loaded on processor 92 when needed to rendercontent, such as a hypertext document, from an ISP or other contentprovider. The browser can be implemented as a hyperlink browser, or moreparticularly, as an Internet Web browser.

Although not shown, the viewer computer might also include on or morestorage drives, such as a CD-ROM drive, to play interactive CD-ROMs inthe event that supplemental data is supplied locally on disk.

FIGS. 6 and 7 show a method for operating an interactive entertainmentsystem to enhance a conventional continuous video data stream, such as atelevision show, with interactive supplemental content. This method isdescribed with additional reference to the screen displays shown inFIGS. 8 a-8 c.

The method begins when a viewer tunes to a particular channel (step 150in FIG. 6). The channel navigator 102 controls the tuner 98 to tune tothe channel. The viewer computing unit checks the appropriate channeland time slot of the EPG data structure 48 to determine if the programbeing carried on the selected channel at this time is interactive (step152). As described above, the presence of a target specification in theEPG data field 58 in association with the program is in indication thatthe program is interactive compatible and that there is supplementalcontent for the program. If the data field is empty, indicating that nosupplemental content exists (i.e., the “no” branch from step 154), theviewer computing unit simply displays the video data stream beingreceived through the tuner 98 (step 156).

A run-time technique can alternatively be used for detecting whether aprogram is interactive compatible. Rather than checking the EPG datafield, the viewer computing unit checks a dedicated channel for theexistence of new supplemental content data. The dedicated channel isseparate from the selected channel carrying the program so that thesupplemental content is received by the viewer computing unit currentlywith the program video data. The existence of a supplemental contentdata stream over the dedicated channel indicates that the program beingreceived on the selected channel is interactive compatible. Thistechnique can be carried out without reference to the EPG listing.

If the program is interactive compatible (i.e., the “yes” branch fromstep 154), the viewer computing unit retrieves the target specificationfrom the EPG data structure (step 158 in FIG. 6). The targetspecification might be a pointer to a memory location at the headend, ora pointer to a memory location on a locally running CD-ROM, or ahyperlink to a target resource located at an independent serviceprovider. The hyperlink browser 106 is loaded onto the processor torender the target resource referenced by the target specification (step160).

At this point, there are several ways to initiate viewing thesupplemental content. One approach is to permit the viewer toselectively activate the interactive mode (i.e., the “viewer activate”branch from step 160). The viewer may know that the program isinteractive compatible by checking a newspaper listing or other programguide. Another way is to have the viewer computing unit display an iconor other indicia on the screen to visually inform the viewer that theprogram is interactive compatible (step 162 in FIG. 6). The icon can beoverlaid on the playing video program in an non-conspicuous manner, likethe closed caption or stereo labels.

FIG. 8 a shows an example screen 200 having a video program 202 playingat full-size on the screen. An icon 204 is displayed at the lower rightcorner of the screen to inform the viewer that the program 202 isinteractive compatible. The icon 204 can be displayed throughout theprogram, or faded out after a set time period. This initial screenrepresents a first mode of interactivity, where the viewer is presentedwith the choice of interactive entertainment or non-interactive viewingof the program. As long as the viewer does not activate the icon 204,the viewer computing unit continues to receive the video content programover the selected channel and display that program alone, without anysupplemental content (steps 166 and 168 in FIG. 6).

If the viewer decides to enter into an interactive mode, the vieweremploys a remote control handset, mouse, keyboard, or other mechanism toactuate the icon 204. This causes the browser 106 to start the targetresource located by the target specification listed in the EPG datastructure (step 170 via the “yes” branch from step 164).

This leads to another approach to invoking the supplemental content.Rather than displaying an icon and waiting for input from the viewer,the viewer computing unit can automatically activate the target resourceas soon as the browser is loaded on the processor (step 170 from the“automatic” branch from step 160).

The target resource contains digital data which supports interactivefunctionality in conjunction with the associated video content program.The digital data defines the supplemental content to enable viewerinteractivity with the video content program. The digital data alsodefines a display layout prescribing how the supplemental content andthe video program are to appear in relation to one another whendisplayed on the screen.

The digital data further defines timing information to synchronizepresentation of the supplemental content with the video content program.This timing information can be implemented in many different ways. Oneapproach is to coordinate the supplemental content to a particularprogram start time, and then carefully measure time from that start timeto synchronize presentation of the supplemental content with theappropriate points in the program. Another approach is to characterizethe program in terms of frames, and key the introduction of supplementalcontent to the frame numbers.

The video stream and supplemental content can be transmitted together,as a single signal, or separately. In the former implementation, thedigital data is packaged with the video stream and transmitted as onesignal from the headend. The viewer computing unit receives the singlesignal at tuner 98 and separates the digital data from the video signal(steps 172 and 174). In the latter implementation, the video data streamis received over the selected channel via tuner 98 from the programprovider (step 172). The digital data is received from the targetresource that is activated by the browser (step 174). This targetresource might be located at the headend, in which case, the data isreceived over a separate channel using tuner 100; or the target resourcemight be served by an independent service provider, in which case, thedata is received over a public network using the modem 100.

At the viewer computing unit, the digital data is deconstructed toextract the timing information and the display layout from thesupplemental content (step 176 in FIG. 6 and step 178 in FIG. 7). Thedisplay layout defines a program boundary within which the program isdisplayed to the viewer. The program boundary is sized and shapedaccording to parameters prescribed in the display layout. As the sizeand shape changes, the processor scales the video data for displaywithin the program boundary at that instance (step 180 in FIG. 7).

The display layout also defines how the supplemental content isillustrated along with the program. The display layout prescribes thesize, style, location, and other parameters for presenting thesupplemental content. For instance, the supplemental content might be atleast partly overlaid on the video program, or provided as a wrapperaround, or along side, the program. The supplemental content isdisplayed according to this display layout and synchronized to theprogram according to the timing information (step 182 in FIG. 7). As anexample, the supplemental content might be a trivia game which quizzesthe viewer as to possible outcomes of various scenes. The questions aredisplayed on the screen according to the display layout and are timedusing the timing information to coincide with the part of the program towhich the questions pertain.

FIG. 8 b illustrates a screen 200 that is part of the interactive mode.Here, the video program is constrained within a program boundary 210which is less than full-screen size. The program boundary 210 has beenreduced in comparison to the full-size presentation of FIG. 8 a to makeroom for the supplemental content. In this illustration, thesupplemental content consists of a main menu having soft buttons212-221, a highlighter or cursor icon 224, graphics bars 226, and a textblock 228 to hold the program title. The soft buttons 212-221 presentvarious control options to the viewer to invite interactive involvementwith the program.

The illustrated screen is taken from an interactive program based on aStar Trek episode. To provide an example of the main menu, the softbuttons 212-216 represent actions for calling up related TV shows andmovies. For instance, button 212 relates to the original Star Trekseries; button 213 concerns The Next Generation series; button 214represents the Deep Space Nine series; button 215 represents the Voyagerseries; and button 216 relates to the various motion pictures.

Soft button 217 is an access/classified toggle which allows selection ofone of two levels of interactivity. “Access” serves as a default mode ofinteractivity which contains the main menu functionality. The screen ofFIG. 8 b is an example of the access mode of interactivity. A voice-overnarrative announces the mode when selected by button 217. Once in theaccess mode, the broadcast video stream is shrunk to fit within thesmaller program boundary 210 to accommodate the main menu graphics. Thealternate mode of interactivity, or “classified” mode, containstime-linked contextual information to provide a second tier ofinformation. FIG. 8 c, shown below, is an example of the classified modeof interactivity.

Soft buttons 218-220 enable selection of different types of supplementalcontent. Button 218 relates to an encyclopedia containing numerous factsand information about the program. In the Star Trek program, forexample, the encyclopedia might contain information on aliens,behind-the-scenes tidbits, cast and crew biographies, facts on otherepisodes, interviews with the actors/actresses, a technical manualexplaining intricacies of starship operation and travel, and varioustimelines.

Button 219 is associated with a program guide, which provides a listingof current Star Trek shows for a two week period. The program guide alsopermits the user to change channels to another channel carrying a StarTrek show, if available.

Button 220 allows selection of entertainment content, which includestrivia games related to the program and a merchandise catalog. Thetrivia games segment allows viewers to compete against other regionalcompetitors. The catalog lists available merchandise related to theprogram that can be purchased by the viewer.

The tenth button 221 is a quit button which allows the viewer to leavethe interactive mode and return to a full-screen display, as shown inFIG. 8 a.

The program and supplemental content are displayed as prescribed by thedisplay layout. The digital data from the target resource, however, candynamically change the display layout. When the display layout ischanged (i.e., the “yes” branch from step 184), the viewer computingunit re-sizes and/or reshapes the program boundary 210 and location ofthe supplemental content to achieve the presentation envisioned by thecontent developer (step 186). The display layout can be changed forvarious reasons. The display layout might change in response to viewerselection of a soft button, as is the case causing the display layoutchange between FIGS. 8 b and 8 c. The display layout might also bealtered automatically as part of the timing information. For instance,the digital data might invoke a graphic or text to pop up on the screenat a timely point in the program. Such real-time content includes, forexample, trivia questions, interesting facts, graphical or soundeffects, and so forth that relate to specific parts of the program.

FIG. 8 c shows the screen 200 at another instance after the viewer hasselected soft button 220 to enter into a classified mode ofinteractivity. The main menu is still present on the screen, but morecontextual information is added in a graphical format to represent anextension of the selected soft button 220. The program boundary 210 isfurther reduced to accommodate the additional graphical and textualinformation.

In this illustration, the viewer has selected the entertainment button220 to display the merchandise store. This layout places the videoprogram box 210 in the lower right hand corner of the screen, and leavesthe remaining area of the screen to be used for supplemental content.The supplemental content comprises a secondary menu having actuatablesoft buttons 232-237. The cursor symbol 224 is movable among the variousbuttons and used to activate a selected button. The buttons correspondto various types of merchandise, such as clothes, posters, collectibles,memorabilia, and toys. There is also an order button 237 that permitsthe user to place an order for a particular product. A picture window240 is arranged beside the secondary menu to show a picture of themerchandise associated with the category indicated by the cursor symbol224. A merchandise description window 242 contains a brief descriptionof the merchandise shown in the picture window 240.

The system and method for presenting interactive entertainment programsis advantageous as it returns the freedom of creativity to the contentdeveloper. By embedding the display layout instructions within thedigital data provided by a target resource, the developer is empoweredto create both the content and the presentation format of how thecontent and broadcast program are displayed to the viewer. The developeris free to control the location and shape of the broadcast programwindow, as well as the presentation format of the supplemental contentrelative to the program window. This facilitates a non-restrictiveenvironment in which the developer can create screens with optimalappeal to the viewer. No longer is the developer simply developingcontent to be displayed within a fixed pane that cannot be dynamicallyaltered relative to the program.

The creative power is thus placed in the hands of the target resourceauthor. FIG. 9 shows a method for authoring an interactive entertainmentprogram. The author begins by developing the supplemental content whichis to accompany a particular television show or movie (step 250 in FIG.9). The author designs the presentation format for each scene of theprogram (step 252), and the timing requirements for synchronizing thesupplemental content with the broadcast video stream (step 254).

In the implementation described herein, the supplemental content isconstructed as a hypertext document which can be rendered by a browser.To control how the content is to be rendered, the author encodes thepresentation format and timing instructions as extension attributes tohyperlink codes. More specifically, hypermedia content utilized by theWWW is commonly written using what is referred to as a “markuplanguage.” “SGML” (Standard Generalized Markup Language) is one suchlanguage, defined formally as “a language for document representationthat formalizes markup and frees it of system and processingdependencies.” SGML is a language for describing the structure ofdocuments and for describing a tagging scheme to delineate thatstructure within text. For creating hypermedia content, WWW documentsutilize a subset of SGML called “HTML” (Hypertext Markup Language). AnHTML textual document can be thought of as plain text that containsformatting instructions in the form of HTML markup codes or “tags.” Tagstell Web browsers how to render and print documents, and are also usedto specify hyperlinks.

The following is a simple example of a start tag for an HTML hyperlink:

-   -   <A HREF=“http://www.microsoft.com/upgrades”>

The leading “A” in the start tag is the name of HTML element. The letter“A” indicates that the tag is an “anchor” tag—the type of tag thatdefines a hyperlink. The start tag contains an “attribute” indicating atarget specification:

-   -   HREF=“http://www.microsoft.com/upgrades”.

The attribute name is “HREF” and the attribute value is“http://www.microsoft.com/upgrades.” HTML documents are organizedaccording to various tags. Tags define the start and end of headings,paragraphs, lists, character highlighting and links, and so on.

The interactive entertainment system described herein supportsconventional HTML documents as target resources. Accordingly, authorscan use familiar HTML elements, such as Head, Body, and Character-Levelelements, to construct a target resource. However, the inventors havealso developed new types of tags, as well as extension attributes toexisting HTML tags. The author of a target resource can use the new tagsand extension attributes to formulate how and when the browser rendersthe supplemental content along with the continuous video stream. The newHTML extension attributes are given below in Table 1.

TABLE 1 HTML Extension Attributes Extension HTML Tag Attribute FunctionBODY background Specifies a URL for an image tile or MMS stream to coverthe document background to yield a distinctive appearance. IMG dynsrcSpecifies the address of a video clip or broadcast source to bedisplayed. FRAMESET usedefault Used to determine where the focus islocated in a tab-based user interface within a FRAME by specifying theURL for the file containing the map, followed by a ‘#’, followed by thename of the USEDEFAULT. If the argument to USEDEFAULT starts with a ‘#’,the map is assumed to be in the same document as the USEDEFAULT tag

The new tags supported by the interactive entertainment system areprovided in Table 2.

TABLE 2 HTML Tags HTML Tag Function Permitted Attributes ACTION Controlsupdate CELL—identifies object in a or display of GALLERY sound orpicture HREF—URL to resource or object ID—label or id for this elementTARGET—where to perform action such as a FRAME or RENDERIMAGE elementEVENT Associates trigger TRIGGER—reference to and action TRIGGER elementto be used with this event ACTION—reference to TRIGGER element to beused with this event - allow comment delimited list of actions tosatisfy the need for multiple actions in response to a single triggerID—name of event GALLERY Stores and SOURCE—URL to actual graphiccoordinates ID—name of gallery collection of ROWS—height of each imagesgraphic cell COLUMNS—width of each graphic cell TRANSPARENCY—RGB valueindicating background color LOADFONT Downloads and HREF—URL to fontresource installs a font to be used RENDERIMAGE Displays an REF—name ofGALLERY image from the CELL—specifies individual cells GALLERYcontaining image TRANSITION Defines screen TYPE—type of transitionchange interim, SPEED—how fast does the fade to black, transition occur,SLOWEST, cut, and so on. SLOW, MEDIUM, FAST, FASTEST TRIGGER Defineswhen ID—name of this TRIGGER something is to TIME—time in millisecondssince occur the last event

Using the supplemental content and HTML tags for timing and presentationformat, the author constructs the target resource (step 256 in FIG. 9).The target resource is stored in a storage medium at a host computer anda target specification for referencing that location is defined (step258). The author submits the target specification to the authorityresponsible for developing the programming information maintained in theEPG database. The target specification is stored in the appropriate datafield of the EPG data structure which corresponds to the program towhich it pertains (step 260).

In compliance with the statute, the invention has been described inlanguage more or less specific as to structure and method features. Itis to be understood, however, that the invention is not limited to thespecific features described, since the means herein disclosed compriseexemplary forms of putting the invention into effect. The invention is,therefore, claimed in any of its forms or modifications within theproper scope of the appended claims appropriately interpreted inaccordance with the doctrine of equivalents and other applicablejudicial doctrines.

1. A method for presenting an interactive program, comprising: receivinga program as a continuous stream of video data; receiving digital datafor supporting interactive functionality in relation to the receivedprogram, the digital data including both supplemental program contentthat enables viewer interactivity with the program and a display layoutprescribing how the supplemental content and the program are to appearin relation to one another, wherein the digital data is received inresponse to user input through an electronic programming guide (EPG)allowing selection of uniform resource locators (URLs) associated withdigital data; displaying the program within a program boundary on avisual display device; presenting the supplemental program content fromthe digital data in a presentation format on the visual display devicewhich enables the interactive functionality; dynamically controllinglocation and shape of the program boundary and the presentation formatof the supplemental program content relative to the program boundary onthe visual display device based on the received digital data, whereinchanges in the location, shape, and relative sizing of the programboundary and a boundary of the supplemental program content are based ontime elapsed since a start time of the program; and presenting both thesupplemental program content and the program simultaneously, wherein thesupplemental program content is presented outside of the programboundary.
 2. A computer implemented method comprising: receiving aprogram from a first source as a continuous stream of video data;selecting a continuous stream of digital data from either the firstsource or from a second source that is different than the first source,wherein the selected digital data supports interactive functionality inrelation to the program, wherein the digital data prescribes howsupplemental content and the video data are to appear in relation to oneanother, wherein the second source is the internet and the continuousstream of digital data is constructed as hypermedia, wherein the digitaldata is selected by providing an electronic programming guide (EPG) to auser for selection from between uniform resource locators (URLs)presented by the EPG to thereby allow user selection between digitaldata from the first source and the second source, and wherein the URLsare user selectable using a back channel to communicate with a headendfrom which the video data is obtained; displaying the program within aprogram boundary on a visual display screen, wherein changes in thelocation and the shape and relative sizing of the program boundary and aboundary of the supplemental content are based on time elapsed since astart time of the program; simultaneously presenting supplementalcontent from the digital data in a presentation format on the visualdisplay screen which enables the interactive functionality; anddynamically controlling location, size and shape of the program boundaryand the presentation format of the supplemental content relative to theprogram boundary on the visual display screen based on the receivedcontinuous stream of digital data, wherein video data is scaled fordisplay with the program boundary in response to dynamic changes insize.
 3. A method for a headend of an interactive television system toauthor an interactive entertainment program, the method comprising:constructing digital data to support interactive functionality with avideo content program, the digital data being configured to permit aviewer to interactively control display of supplemental content alongwith the video content program, wherein the digital data is presentedaccording to user input at an EPG (electronic programming guide)allowing selection between URLs (uniform resource locators) within theEPG; defining a display layout of how the supplemental content and thevideo content program are displayed in relation to one another on adisplay device; developing timing information to synchronizepresentation of the supplemental content in conjunction with the videocontent program wherein the timing information coordinates thesupplemental content with a start time of the video content program andwherein the timing information is derived from the supplemental content;and encoding the digital data with instructions to dynamically changethe display layout of the supplemental content and the video contentprogram and to alter the display layout of the supplemental content andthe video content program in response to the timing information toaccommodate the display of both the supplemental content and the videocontent program on the display device, wherein changes in location,shape, and relative size of a boundary of the video content program andthe supplemental content are keyed to frame numbers of frames beingdisplayed.
 4. A computer implemented method, comprising: constructingdigital data to support interactive functionality with a video contentprogram, the digital data being configured to permit a viewer tointeractively control display of supplemental content along with thevideo content program, wherein the digital data is presented in responseto user input to an EPG (electronic programming guide) and userselection of URL (uniform resource locator) within the EPG; defining adisplay layout of how the supplemental content and the video contentprogram are displayed in relation to one another; and encoding thedigital data with instructions to dynamically change the display layoutof the supplemental content and the video content program automatically,wherein the encoding includes changing a size of the video contentprogram displayed on a screen and a size of the supplemental contentdisplayed on the screen, and wherein changes in the location andrelative sizing of a boundary of the video content and a boundary of thesupplemental content are keyed to frame numbers of frames beingdisplayed.
 5. The method of claim 1, wherein changes in the location,shape, and the relative sizing of the program boundary and a boundary ofthe supplemental program content that are based on time elapsed sincethe start time of the program are based on an open loop control, whereinthe start time is followed by measurable ticks.
 6. The method of claim1, wherein changes in the location, shape, and the relative sizing ofthe program boundary and a boundary of the supplemental program contentthat are based on time elapsed since the start time of the program arebased on an open loop control, wherein the start time is followed by aframe count.
 7. The method of claim 1, wherein receiving digital datacomprises presenting a user with an opportunity for selecting acontinuous stream of digital data from either a first source or from asecond source that is different than the first source, wherein the firstsource supplies the program.
 8. The method of claim 1, whereindynamically controlling location and shape of the program boundary andthe presentation format of the supplemental program content relative tothe program boundary on the visual display device comprises scaling thevideo data.
 9. The method of claim 1, wherein the EPG determines whetherthe program has associated digital data by reviewing for presence ofassociated URLs, and wherein a small icon is presented indicate digitaldata is available when appropriate.
 10. The method of claim 3, whereinchanges in location, shape, and relative sizing of the boundary of thevideo content program and a boundary of the supplemental content arebased on a measurement of ticks following the start time of the videocontent program.
 11. The method of claim 3, wherein changes in locationand shape and relative sizing of the boundary of the video contentprogram and a boundary of the supplemental content are based on a framecount following the start time of the video content program.
 12. Themethod of claim 3, wherein the digital data is constructed in responseto selection of digital data from either a first source or from a secondsource that is different than the first source, wherein the first sourcesupplies the video content program.
 13. The method of claim 3, whereindynamic change to the display layout of the supplemental content and tothe video content program comprises scaling video data of the videocontent program.
 14. The method of claim 3, wherein constructing thedigital data comprises identifying whether the video content program hasassociated digital data by reviewing for presence of associated URLswithin the EPG, and wherein a small icon is presented to indicateavailable digital data when appropriate.
 15. The computer implementedmethod of claim 4, wherein changes in the location and the relativesizing of the boundary of the video content and the boundary of thesupplemental content that are based on time elapsed since the start timeof the video content program are based on an open loop control, whereinthe start time is followed by a frame count.
 16. The computerimplemented method of claim 4, wherein constructing digital datacomprises presenting a user with an opportunity for selecting acontinuous stream of digital data from either a first source or from asecond source that is different than the first source, wherein the firstsource supplies the video content program.
 17. The computer implementedmethod of claim 4, wherein dynamically changing the display layout ofthe supplemental content and the video content program comprises scalingthe video content program.